Speech recognition devices are increasingly popular. A speech recognition device may be used under different circumstances or in different environments. For example, a user may ask the device a question while working at a cubicle with other co-workers surrounding the user. A user may also ask the device a question while attending a meeting in a conference room with other meeting participants. A user may also speak to the device while studying in a library where speaking loudly may be prohibited. Under some circumstances, the user may whisper speech to the device and/or desire that responses be whispered. It is thus important that a device recognizes the user's whispered speech, which may vary from normal speech, and provide a whispered speech response under these circumstances.